Using Collocations to Assess MT Quality

Conventional metrics for Machine Translation evaluation have focused on using n-gram similarity between a reference translation and a system translation as an indication of the system quality. A simple n-gram model however cannot capture long-distance dependency, and the requirement of a reference translation has prevented the use of these metrics at the decoding stage. In this paper we propose a set of collocation-based metrics to address these problems. A series of experiments has shown that these metrics are capable of distinguishing between human translations and system translations, and can produce system rankings comparable to the other metrics without requiring a reference translation.